Analysis

Data preparation

This is an automatically generated report using the data from the survey about experiment issues.

In the tabs below you can see tables, descriptive plots, analysis, etc.

So far, 204 people completed the survey.

The last time the data was updated was on 10/11/2023.

Descriptive tables

Table 1. Frequency of issues and GodLike behaviour
Characteristic N = 2041
Frecuency of issues
    Never 35 (17%)
    Rarely 44 (22%)
    Sometimes 50 (25%)
    Often 39 (19%)
    Always 36 (18%)
Who is God
    Me 15 (43%)
    My research assistant 20 (57%)
    ? 169
Experience
    Student 44 (22%)
    Postdoc 58 (29%)
    Early Career 61 (30%)
    Senior 38 (19%)
    ? 3
1 n (%)
Table 2. Frequency of types of issues. 10 most common issues
Issues in experiments N = 9531
Issue
    Other 94 (9.9%)
    Errors so bad that my friend had to start over 90 (9.4%)
    Important content errors 90 (9.4%)
    Data preparation was harder than it could have been 88 (9.2%)
    Some variables or questions asked to participants but not used in the analysis/paper 88 (9.2%)
    Match between hypotheses and data not clear 86 (9.0%)
    My friends do not have errors in their protocols 85 (8.9%)
    Errors in the coding of items 84 (8.8%)
    Data not what we expected 83 (8.7%)
    Spelling errors 83 (8.7%)
    Errors in participants balancing to conditions 82 (8.6%)
1 n (%)
Table 3. Frequency of types of issues by Expertise.
Issues in experiments 1, N = 441 2, N = 581 3, N = 611 4, N = 381
Frecuency of issues



    Never 6 (14%) 14 (24%) 10 (16%) 3 (7.9%)
    Rarely 11 (25%) 12 (21%) 12 (20%) 9 (24%)
    Sometimes 12 (27%) 10 (17%) 15 (25%) 12 (32%)
    Often 9 (20%) 11 (19%) 10 (16%) 9 (24%)
    Always 6 (14%) 11 (19%) 14 (23%) 5 (13%)
1 n (%)

Plots

Figure 2: Types of issues

Figure 2: Types of issues

Analysis

We fitted a linear model (estimated using OLS) to predict Number of Issues with
Frequency of Issues and Experience (formula: `Number of Issues` ~ `Frequency of
Issues` + Experience). The model explains a statistically significant and weak
proportion of variance (R2 = 0.12, F(2, 198) = 13.45, p < .001, adj. R2 =
0.11). The model's intercept, corresponding to Frequency of Issues = 0 and
Experience = 0, is at 2.76 (95% CI [1.38, 4.14], t(198) = 3.94, p < .001).
Within this model:

  - The effect of Frequency of Issues is statistically significant and positive
(beta = 0.93, 95% CI [0.57, 1.28], t(198) = 5.16, p < .001; Std. beta = 0.34,
95% CI [0.21, 0.48])
  - The effect of Experience is statistically non-significant and positive (beta
= 0.05, 95% CI [-0.41, 0.51], t(198) = 0.20, p = 0.839; Std. beta = 0.01, 95%
CI [-0.12, 0.15])

Standardized parameters were obtained by fitting the model on a standardized
version of the dataset. 95% Confidence Intervals (CIs) and p-values were
computed using a Wald t-distribution approximation.
Characteristic Beta 95% CI1 p-value
(Intercept) 2.8 1.4, 4.1 <0.001
Frequency of Issues 0.93 0.57, 1.3 <0.001
Experience 0.05 -0.41, 0.51 0.8
R² 0.120

Adjusted R² 0.111

Residual df 198

No. Obs. 201

1 CI = Confidence Interval

Figure 3: Model plot

Figure 4: Descriptive plot

Anything else

  • Automatic reports per participant
  • Automatic emails to participants
  • Fully reproducible pdf version of article using journal templates
  • Etc.